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C882/A 

TITLE : — Method for Con volution .of SignaliorrlmaaerData 
Field of the Invention 

This invention relates to a method of convolving, an input, data 
set with a further data set to produce an output data set. The 
method is particularly, but riot exclusively, applicable to 
image processing in which a data set constituting an image is 
convolved with a further data set (which may act as a filter) . 

Background to the^lnvention ^ 

Convolution is a useful computational technique/ which- is used 
in many signal and/or. image processing ^applications . For all 
combinations of two data elements, one from each set , the 
products of the values of the two data elements are found and 
the positions of the elements in the two. sets are used to map 
the resultant data into ah output data set. The technique may 
be used, for example., to simulate certain photographic effects 
in a computer generated image, for example lens blooming and 
internal reflections . 

Certain methods directly find the products of all possible 
combinations of data from the two- set's. . . Theses methods can-be 
very slow particularly for large data setes, as -would generally 
be. involved ^with image- processings 

The convolution process can be accelerated using Fourier 
Transform methods by which the data sets are transformed into 
the frequency domain and then convolved, for example by the 
Fast Fourier Transform (FFT) method.. However., such. techniques, 
can still never thel ess be time consuming g and can require a 
considerable amount . of temporary data storage to be available 
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in the apparatus performing the convolution. 

It has also been found that even if one of the data input sets 
is sparse (i.e. has lots of zero-valued, elements or elements, 
with very small values) either of the above techniques can 
still be time consuming. 

Background to the Invention 

According to the present invention, there is provided a method 
of convolving an input . data set with a further data set, each 
of the sets containing a plurality of data elements performing 
convolution calculations mathematically to combine the values 
of the data elements of the two sets and to map the results 
into an output data set, the method comprising the steps of : 

(a) Arranging . the elements of each of the input and 
further data sets into a respective set of groups ; 

(b) For each group, obtaining an average of the values 
of the data elements contained in that group and obtaining at 
least one variance value representative of the range of values 
of those elements; 

(c) Determining for which, if any, pair combination of 
groups, one from .each, set, . the results . of convolution 
calculations based on average values for at least one of the 
groups in the pair would", not differ significantly from the 
results of convolution calculations based on the individual 
value's of the data elements contained in both groups of the 
pair, and using the average value or values in the convolution 
calculations for the combinations of data elements for those 
pairs by treating, the values of all the data, elements in the 
or both groups as being constant and substantially equivalent 
to the average value for their group; . 



(d) Basing the convolution calculations for the remaining 
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pair of combinations of groups, if any, on the individual 
values of data elements contained therein.. 

Thus, the invention determines whether theveiements contained 
in a given group, can be represented as having the^same value 
for the purposes of the convolution calculations.- If they can, 
this reduces, the amount of computation need to perform the 
calculations since, the individual data values from ■ the group 
concerned do not have to be read at least for those 
calculations- Furthermore, if both groups in a given pair can 
be represented by average, values/ the product value only has 
to be calculated once in the convolution of the data from those 
two groups . 

The invention thus provides* a way of finding suitable 
approximations for values of certain data elements in order to 
reduce the amount of data which has; to -be^pgpcessed in the 
convolution calculations. The invention therefore^ achieves an 
approximate convolution to > an^ac cep tab ie - leve&v^ofkr accuracy . 

Prefer-ably, the input datea set- comprises- a representation of 
a perceivable subject and may thus, for example, form, at , least 
part of. an image or an audio signal. However, the invention 
is particularly applicable to an input- data set representative 
of an image (each data element corresponding to a respective 
image pixel, with the value of that element corresponding to 
colour or brightness of the pixel) . Thus, the invention can 
provide a method of processing an image signal by convolving 
the image data carried thereby with . a further data set - 



Preferably, each set of groups is hierarchically arranged, with 
the members of each group, .at the bottom level .of each 
hierarchical set being constituted by the datea ^elements .of that 
set, and the members of the or. each group in the- or each higher 
level being* constituted -by groups from the level below, so 
that, the higher a group, in. the hierarchy, the larger the 
number of data elements it contains (either directly or. through 
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containment of other groups from lower levels) . 

Preferably, in step (c) the groups of each hierarchical set are 
considered in descending order so that any given pair- of groups 
as analysed in accordance with step (c) only if it has been 
determined that average values for the higher level groups. 

which contain the members of that given pair, are not to be 

used . 

This provides an effective way of determining whether average 
values can be used in respect of large proportions of the data 
sets at a relatively early stage of the convolution method. 

If the input data set constitutes ah image, the groups can 
correspond to regions and sub-regions (each contained in a 
region) of the . image so that the higher the group in the 
hierarchy the larger the -region it represents. 

Said representative variance values for any given group may 
comprise the difference between the maximum and minimum values 
of the data elements contained therein, in which case it can 
be determined whether the elements contained in the group can 
all be treated as being of the same value, equivalent to their 
average, for the convolution calculations if the difference, is 
less than a given threshold. . 

Preferably, however, the representative variance values for 
each group comprise the maximum and minimum values of data 
elements contained thereby, and the elements of each group in 
any group pair combination are all treated as having the same 
value (i.e. the average value) if the difference between the 
products of the maximum values and the products of the minimum 
values of . the two groups is below a given threshold. 

In the case of image data, the threshold may be that value 
below which there would be no perceivable, variation in the 
portion of the output, data created by convolving the two 
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groups . 

Preferably, for image data, the method includes the step of 
modifying said threshold, after the . data sets have been 
partially convolved, independent on the portions of the output 
data set already created by the partial convolution. For 
example, the threshold may be raised where the output for a 
given group is to be situated adjacent a portion of the output 
image which has already been determined and is relatively 
bright, or is of. maximum brightness. 

Preferably., for each pair combination of sets for which it has 
been determined that convolution calculations based on average 
values would not differ significantly from calculations based 
on the individual values of the data elements contained in 
those groups, the method . includes a further step of treating 
the value of all the elements in both groups as zero if the 
products of the averages for the two groups is below a given 
threshold* 

Preferably, this threshold can also be modified in response to 
output data already obtained in. a similar fashion to that in 
which the threshold which is applied in relation to the 
variance values can be modified . 

If only a region of the output data set is required, the method 
preferably includes the additional step of determining which 
of the groups contain data elements, the convolution 
calculations on which will produce a result outside said region 
in the. output data set, and discarding those groups . 

Preferably,, this step is performed before step (c) above. 

It will be appreciated that the method of the present invention 
can significantly expedite convolution of two image data sets 
where at least one of the sets has relatively large areas of 
uniform brightness/colour (depending upon the values being 
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convolved) or large areas of zero intensity) . Also, the 
discarding of certain groups as discussed above can 
significantly reduce the amount of processing-*reqiti r ed *f or the T 
convolution, and is a step which, is not possible^where ; Fourier 
methods are used, since the latter require ^the^transf ormat ion 
into the frequency-domain of both the input and' -further- data 
sets in their entirety. 

Brief Description of the Drawings 

The invention will now be described, by way of example only, 
with reference to the accompanying drawings, in which: 

Figure 1 is a schematic diagram showing two one -dimensional 
data, sets (in. simplified form) which- may be convolved by a 
method in accordance with the invention; 

Figure 2 ; is -,a schematic diagram shoeing .how two * other rone - 
dimensional , data sets , (which are to be convol-ved-Tin ^accordance 
with, the present invention) . can be* arranged into two 
hierarchical eeta of groups;* 

Figure 3 illustrates the contribution made by the convolution 
of selected portions of the . two data sets in Figure 2 to the 
output data set for the convolution; 

Figure 4 illustrates data which is stored in respect of each 
of the groups illustrated in Figure 2; and 

Figure 5 is a diagram (similar to Figure 1) showing three data 
sets, one having been formed by the convolution of the .other 
two. 

Detai ledtiPeseriptiori " 



Figure 1 shows two data sets, represented as two columns of 
boxes. One of those, columns ■, column generally referenced x. 



2^^99 FRI 14:27 KEITH l^H & Qo. FAX NO, 01223 ^53 ^ _ P. H 



represents an input data set fox* a line of an image. Each 
individual box in the column represents a pixel . in that line, 
and the numerical value contained in the box is representative 
of the brightness intensity level of the pixel, whilst the 
position of the box in the data set is related to the position 
of the pixel in the image line .It will be- appreciated that 
the data set has been greatly simplified by only showing a 
relatively small, number of pixels.. In reality., the line of the 
image would contain several hundred pixels, and a column with 
a corresponding number of boxes would be needed: to represent 
such a line. 

Column 2 represents a further data set which acts as a kernel 
of the convolution to be performed on the input data set. Like 
the input data set 1, the kernel corresponds to a line of an 
image of a "filter" to be applied . to the data set 1. The 
numbers inside the boxes in the column of the data . set 2 
therefore correspond to. pixel Intensities, as is the case with 
the input data set . However, the relative position of each box 
in the. further data set 2 provides a displacement function 
which will be discussed in more detail in relation to Figures 
2 and 3 , . 

As with the input data set 1, the. data set 2 has been greatly 
simplified and would, in reality, have a much larger number of 
individual elements 

The convolution function involves obtaining the products of 
each data element of the input data set i with all the data 
elements of the data set 2, mapping those products into 
positions in an output data set: If a plurality of products 
are. mapped into the same position in the output set, their sum 
is also found. With a large number of data elements in both 
the sets,, performing theae calculations in. respect of all the 
individual data element pair combinations would require a very 
large number of individual calculations to be performed. - 
However, the convolution process can be considerably c:>cp<-d>i.t^dL 
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by using an approximation of the data element intensities 
whereby certain groups of data elements are treated as having 
the same intensity, equivalent to a repjsesewfcative, "average «■ 
value.- 

Accordingly, the method of the present invention commences by 
sorting the data elements into a hierarchical structure of 
groups, and Appendix 2 is a pseudo program code for 
constructing such a hierarchy. 

Figure 2 shows an input data set 4 and a further "filter- data, 
set 6, both of which constitute single lines in a respective 
image. These data sets are represented as rows of boxes, each 
of which is a data element corresponding to a respective pixel. 
The numbers appearing in the boxes represent the intensities 
of the corresponding pixels, whilst the numbers set out. beneath 
the boxes (Nob 0-15 for set 4 and. -7 to 8 >f onsets*. 6) are the 
positional., data for the individual dat-a elements . In reality, 
a far. larger number of..pixei«s,* and heneesdat 1 a-'-eiement©,- would 
be present. 

The individual data elements of the set 4 are arranged into a 
hierarchical set of groups which is depicted by means of a 
binary-tree generally referenced 8. whilst a similar binary- 
tree generally referenced 9 depicts the grouping of the data 
elements of the set 6 into a hierarchical group structure. 

Each node of each tree represents a respective group, and the 
connections between the groups indicate which groups contain 
which. The leaf nodes of the tree (i.e. the bottom of the 
tree) represent the individual data elements. Thus, the 
individual data elements are at level 4 of the hierarchy, and 
each of the groups in level 3 .contain a respective pair of data 
elements. For example, the node> io represents ^a *group. which 
contains, data elements at positions o and 1 in the set 4 . The 
members of each node in level 2 comprise a respective pair of 
adjacent nodes from level 3, - so that the nodes at level 2 each 
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contain a respective set of four consecutive data elements. 

Thug, the group depicted by the node 12 contains the groups 14 
and 16 from level 3, .and thus also the data elements at 
positions 4-7 inclusive. 

It can be seen from the diagram that each group at level 2 
contains pixels which correspond to a respective region 
constituting a quarter of the image line. The group 

represented by each node at level i contains a respective pair 
of groups represented by the nodes at level 2 , and therefore 
corresponds to half of the image line. Thus, for example, the 
group represented by the node 18 contains those groups 
represented by the nodes 12 and 20. The tree 8 has a root node 
22 which corresponds, to the whole of the image line. 
Similarly, each group at level 3 of the .structure represented 
by the tree 9 contains the elements for a respective eighth of 
the image, a respective quarter of the image is contained in 
each group at level 2 , half the ; image • in each group, at level 
1 and the whole filter image at the group represented by root 
node 24 , 

For each node, there is obtained the following values which are 
representative of * the. data elements contained by the group 
represented by the node s 



.1. " The average value of the intensity of the elements (in 
this example the average is the mean average and the average 
values., for the. nodes at levels 0, l and 2 are obtained by 
taking the mean average values of the relevant nodes from the 
level below) ; 

2 . The intensity value of the data elements with the 

maximum intensity value.; 



. 3. A minimum intensity value equal to that of the element, 
with the minimum intensity; 
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4. The start position of the data elements, (the position 
of the left-hand data element in this example); 

5 . The length of the region represented^ljy the data 
elements 

The average intensity values have been written into the boxes 
indicating individual nodes in the tree shown in the Figures. 
Thus, for example, the average intensity associated with the 
node 20 is .1, the maximum intensity is .2, the minimum 0, the 
node start position is 0 and the node length is 4. The data 
stored for each node are represented in Figure 4 and also 
includes data identifying the child nodes (or sub-nodes) i.e. 
those nodes (or data elements if the node is in level 3) from 
the level below, to which the node is connected. The maximum 
and minimum values stored in respect, of the- nodes -constitute 
the -representative variance values which -axgs dise.d..to -provide 
a measure of the variation in ■*th&,v a .lues«whdch the^convolying 
of the data elements concerned would contribute to the output . 
Depending on the use , of the output image- a -threshold value, 
referred to as TV, can be. calculated such that TV is the 
largest change in . intensity of picture elements in the output • 
image to. which a human viewer is insensitive when the image is 
represented on an output display. 

The convolution of any given pair combination of individual 
data elements of the sets 4 and 6 involves taking the products 
of the intensities of those, elements, and mapping that product 
into an output data domain part of which is shown at 26 in 
Figure 3. The address of the output domain into which the 
product is mapped is determined^by^taJci-ng- the- position of the 
relevant data element from -the .sets* 4 and.twdispsLiaeing.it by an 
amount determined by the position data for the element in the. 
set 6, For example, the convolution of the data elements 28 
(from set 4) and 3 0 (from set 6) will give a product of 
1.0x1=10, and this will contribute to. the value at the address 
5 (i.e. 5 X 7) in the set 26> The final value at the address 
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5 of the set 26 will be the sum of all the products which are 
.mapped to that address. The product contribution for the 
elements 28 and 30 is indicated by the notation T (4 , 5) *F (4 , y) , 
and a similar notation is used to indicate the contributions 
of other nocie pairs in the data. sets. The notation can be 
generally represented as I (a,b)*F(c,d) where: 

I indicates the input data set 4, 

F indicates the filter data set 6, 

a and c represent the respective levels (0-4) of the 
node, and 

b and d represent the number of places (from 0 to the 
total number of nodes at the level -1) from which the 
node is spaced from the left side of the level.. 

Some .products. will be mapped into other addresses in the data 
set, but these arc not of interest (for example because they 
lie outside the required output image) . 

The procedure by which the method determines which of the. 
elements can be represented by average values in the 
.convolution calculations Will now be described. 

Firstly, the two root nodes 22 and 24 are compared by 
calculating the difference between the product of the maximum 
element values of the nodes 22 and 24 and the product of the 
minimum element values for the nodes. This is then compared, 
with the threshold TV. if the difference is less than the. 
threshold, only at most two average values, one for each set 
would be needed to represent the intensity values of all the 
data- elements,. .In the present example, the threshold is set 
at an appropriate level between 1 and 9, and is exceeded by 
said difference, which iri this case is 10. 



It is then determined whether the data elements represented by 
the two nodes, would only contribute to the portions of the 
output data set outside the zone of interest (address is 0-15) , 
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This is achieved using the node start positions and lengths, 
and since the nodes 22 and 24 represent the whole of the data 
sets. 4 and 6, all of »the addresses*$of • intereae in . the. ^output 
data set 26 will be .contributed to by products ©f -data elements 
represented by the two . nodes . 

Accordingly, the comparison of the nodes- 22 and 24 in this case 
will determine that intensity values cannot be approximated, 
nor can combinations be disregarded at this level , and the 
comparison of nodes proceeds to the next stage. 

This involves the comparison of the node 22 with each of the 
two nodes 32 and 34 in the tree 9 in the same way as discussed 
above . 

In this case, the comparison will determine that neither 
average values fdr . the elements ..rep resented ihy the node 
combinations (22 Va and 32 ., , 22 Wsj* > can bemused ^no* would the 
contributions -of the^rc^uc^^a^xs^^'hose^d^fea^elemenfce--. lie 
wholly outside the- addresses- of^in&e-rest in -the Aoutput set. 
If this were not the case;- however/ then the ^comparison with 
all of the sub-nodes of the node pair in- quesfeion need-not 
occur since average- values can be used for all of the data 
elements, or of the product contribution of the. pairs of those 
elements do not have to be considered. 

Accordingly, if, for.' example, all of. the^ data elements 
represented by. the node 32 were of zero value, then a 
comparison of the node 22 (or any of its sub-nodes) with any 
of the nodes 36, 38, 40, 42. 44. and 46 is not required. 

This process ; continues, until the- node. 22 h»s**beera compared 
against all .the necessary nodes-sin the 4ree«-9 '(including the 
data elements.--- in the- «sete*:6) . 



Each node in the- level 1 is then compared with those nodes 
(starting from the top) of the tree 9 which Have not already 
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been ruled out by the previous comparison (s) , and this process 
is repeated as necessary for all the other nodes in the tree 
8, so that each node in the tree 8 is compared against those 
nodes in the tree 9 which have, not already been ruled out from 
further comparison (as a result of the comparison with higher 
level nodes in the tree 8} .. ' . 

An alternative threshold test to determine whether average data 
element values can be used in the convolution of data element 
groups represented by two- giVen nodes (Nodel , Node2) is as 
follows: 

pyramidMultiplier x (maxNodel x maxNode2. - meanNodel x 
meanNode2) < Thresholdl •• 

Where "pyramidMultiplier" is the length (i.e. the number of 
data elements) of the shorter of the two nodes, or the length 
of both nodes if they contain the same number of elements, 
maxNodel/2 is the maximum value of any data element contained 
in the node and meahNbdel/2 is the mean value of the data 
elements in. the node. 

If the comparison, of a given node with another node or data 
element determines that the elements represented by said given 
node can be represented by an average value, it is then 
determined whether that average value can be approximated to 
'zero . ' ■ 

Using the notation mentioned above, this can be done with the 
following formula.: 

meanNodel x meanNode2 x pyramidMultiplier .< threshqld2 

It will be appreciated .that,. i.f either Nodel or Node2 is a 
single data element, its mean will be the value of that data 
element,, whilst -the pyramidMultiplier will be i., 



In the present example, threshqld2 is 0.12. 
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However, this approach ie not university correct as it does not 
consider the worst case scenario in which all the highest value 
data elements in. the., nodeXs.) conspire to contribute , to the 
highest valued output contribution. Thus.,., anwalterriative 
approach would be to apply the* formula: 

MaxNodel x maxNode2 x pyramidMultxplier < th^esfooard 

If zero can be used as an average value for the convolution of 
one group of data elements' with another, then that convolution 
merely involves mapping the zero data values from the 
convolution of those two. groups into the appropriate locations 
of the output data set. 

In the example shown in Figure 2 some of the data elements can 
be represented by average values , . in spme^caee^s zero/ whilst 
certain other data elements have to be convolved directly. 

For example,^ the comparison ^of : the^nod^s^50^and^i2; Reveals not 
only -that .the variance ^ of the -da ta^e lement s ^represented by 
those anodes (i .e the .elemenfe-s^at 12 and -13 of the input data 
set and those at -7 to 0 ^f^the- filter sets) is below the given 
threshold, but also- that' the. average for the node 50 is 0 so 
that -the ^product - of >the . data elements . 12 and 13 in the input 
set . with the data elements -7 to o in the filter set will 
always be G, and it. is thus not necessary explicitly to perform 
the product calculation^ Thus, in Figure 3, the convolution 
•product 1(3,6) *F(1, 0) resulting from the convolution of the 
nodes 5 0 and 32 is shown as. a series of 0 values which start 
at address 5 and extend to address 15. 

I (2,0) *F (2,3) is a case where the node pair product - (for the 
nodee 20, 52) is approximately 0, i.e. less than, the prescribed 
threshold value. Again, . the convolution ^of-the.dafea elements 
represented by those two nodes^ does - not contribute to the 
output v value^of* ! the**dafea set- 2 6.. 



Convolution I (2,2) *F(2, 0) is the case where the v^lvx<-o 
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represented by a node pair 54 and 3 6 are constant eo that they 
can be represented as single values (average values would also 
be applied if the data elements contained in each node were not 
of a constant value, but exhibited . sufficiently little 
variation in value) . In this case, the output contribution can 
be calculated using the same product value for each data 
element combination. As can be seen . from Figure 3, the 
convolution., of those two nodes produces a data set in which the 
values rise from a minimum of . 3 to a peak of 1.2 and then 
descends back to a minimum of .3. The reason for this 
variation is that the product of some of the pairs of data 
elements map. into the same addresses in the output data set and 
therefore accumulate.. For example, the. elements 58 and 60 
provide a product which is mapped into the address 62 as do the 
elements 64 and 66. 

I (2,3) *F (2,3) , produces a product which is out of range with 
respect to the domain of the output data (since the lowest 
address into which the results of that convolution will he 
mapped is 17), and this ..convolution can therefore be 
disregarded. 

1(4,5) *F (4,7) is a case where, due to high local variance in 
both input and filter nodes, we have to traverse all the way. 
down, to the leaf nodes so that the individual data elements 
need to be convolved' in a conventional way 1 . 

It Will be appreciated that, although the method has been 
described in relation, to one -dimensional data sets,, it is also 
applicable to other data sets, for example data set defining 
a two_ dimensional image and filter. 

A description of a further example of a method in accordance 
with the invention is set out below. 

As before, input data are transformed into hierarchical . form 
and the convolution then proceeds by the following Recursive 
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Decent Procedure (RDP) . The RDP is initially executed with the 
highest node in the first input- data set referred to as Node 
A and the highest node in the second.-iriput^data set^ referred 
to as Node B, according ,to the following steps z 

.1. If the absolute product of the*>Nodel Average element 
Values of Node A and Node B.will produce a total result in the 
output smaller than the threshold value TV, then in this case 
the execution of the procedure is complete which has the effect 
ttiat Node A is not subsequently tested with sub-nodes of Node 
B and- Node B. is not subsequently tested with sub -nodes of Node 
A, 

2. If the position and: length of Node A and Node B are 
such that the result of their convolution would. only change the 
intensity of elements which are less than the minimum position 
in the output data set or greater than the maximum position in 
the output data set (that ' is ^the . domain of the output of the 
convolution^ would not overlap.^the^donuin of the .output data 
set), then in this ^case the execution of the procedure is 
complete which has the effect that Node A is not subsequently 
tested with sub-nodes of Node. B and Node B is not subsequently 
tested with sub-noc|es .of Node A, 

3. If the ' difference in intensity between the Node A MAXEI 
and the Node A MINEI is less, than TV and . the difference in 
intensity between. the Node B MAXEI and the Node B MINEI is less 
than TV, then in this case . Node A and Node B are convolved 
directly by a suitable known method for the convolution of 
cbnistani areas, for example as described by the pseudo program 
code in Appendix 3 using the simplifying assumption that - the 
intensity of the elements in. the regions described^ by. Node A 
and Node B are constant, after having convolved an 
approximation of the* Node A and Node B imatge data t he- execution 
of the procedure is complete wMch,has,.the effect that Node A 
is not subsequently tested with sub-nodes of Node B and Node 
B is not subsequently tested with $ub-nodes of Node A. 
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4. If the Node A Node Length is greater than, the Node B 
Node Length then in this case the; . RDP is executed first with 
the lower, position sub-node of Node A and Node B, - subsequently 
with the higher position sub-node of Node A and Node B. 

5. If the Node B Node Length is greater than the Node A 
Node Length then in this case the RDP is executed first with 
the lower position sub- node of Node B and Node A, subsequently 
with the higher position sub-node of Node B and Node A. 

5- If the Node A Node Length and the Node B Node Length 
are the same and the Node A Node Length is greater than 1 (i.e. 
Node A is not a sub-element leaf Node) then in this case the 
RDP is executed first with the. lower position- sub-node of Node 
A and Node B subsequently with the higher position sub-node of 
Node A . and Node B . 

• if neither candidate pair is rejected or the convolution is 
performed directly, then each candidate is not tested further 
with sub-nodes of the other candidate... This procedure is 
achieved by a suitable method, for example as described in the 
pseudo program code in Appendix. 4. 

V. Once all invocations? of the RDP arc complete the 

output pixel elements Have been calculated and the convolution 
is complete. . ■ 

A sample implementation of the method according to the 
invention written in the. C programming language is shown in 
Appendix 1 arid the effect of this implementation on two example 
input lines in Figure 5, 6.1 and 6.2 is shown at 6.3. 

An extension of the method in accordance with the invention 
would be to apply. the. . method . to data with higher 
dimensionalities for example to the convolution of 2D images 
represented by the known quad- tree representation or to the 
convolution of data^ sets with differing dimeneionalitieB, for 
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example line and 2D image convolution. 

A further extension of the method in accordance with the 
invention , would be to test candidate pairs in» the ^convolution 
for their effect on the image with respeet to , the output 
intensity already accumulated in the output, image , For 
example, if the output at a given location in the output image 
was fully saturated (that is the output intensity has reached 
some notional maximum value beyond which additional intensity 
is not usefully represented) then the candidate pair could be 
rejected at that stage. Similarly the threshold which 
determines whether a candidate pair is important in. the output 
can be varied depending on the output value already 
accumulated, so that for example if. the output at a given 
location is already at high intensity convolutions which would 
provide a relatively small addition in intensity to that 
location are rejected. 

A further-.; extension of the - method i would >* allow, differing 
representat ion of -the - data- afe eaeh level in • the hierarchy and 
during a pre-process that representation which produced regions 
with the lowest variance would be selected. For example, when 
performing a 2D convolution a two-way sub-division could be 
used so that each non-element node has two sub-nodes describing 
two halves of a square, wherein the axis of division (e.g. 
vertical or horizontal) is chosen to minimise the variance of 
the intensities described by. said sub-nodes. 

In a further extension of the method the output image could 
also be represented hierarchically so that determining the 
average intensity already 'accumulated in a region can be done 
efficiently. 

The methods -according to the invention place no-'restKriction on 
the exact - size of the data sets and can, . peasf prm-nontperiodic 
convolution without, explicit zero-padding of the data. This 
is generally hot the case for Fourier baaed methods . it is 
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also worth noting that .methods according to the invention are 
based in the (mathematically) real domain whereas Fourier based 
methods are inherently based in the complex domain. 



U-UJ-ao \*:tL 314*01 P. 14 K-OWO JOD-24S 

12-MaB-99 FRI 14:31 KEUM NASH & Co. FAX NO. QlJBt 324353 P. 24 



NASH & Co. FAX NO. 01^ 

ART Ltd, Company Confidential. 1999 



Appendix z . 

int nusnber_df_,elements = 8; 
int nvunbei : _of_levels « 4; 
int element_poijiter; 

float input^liiiel.LO.. 0 , .o V.0 . 0-0,. 5.0. 4.0, 1 . O , l.O, 1 01; 

float input^llhe^Cl.O, 2.0. 2.0, .1.0. 1.0. 1.0. 1.0. l,0lV 

float otitput_line[«0.0, O.o f 0.0. o;6, 0..0, o.;0,^oVo, o.;03 ; 

typedef struct- nod^_elemeiite- { 
int position; 
int length;- 
float max_value .- 
flo^t miiuvaluo / 
float averagejvalue; 
node_element 'childl: 
node^elcwent *enild2; 
>.."*' 

nodo^element *maKe_node ( int number, int position) { 
nodc_element *node ; 

node = alloc (sized? (node^oiement) ) ; 
node~>position ~ position; 
node->lenerth = length; 
if (number =-= 2) . < 

if (inpufe^Une [position! > input^lin&tpositipn^l] ) £ 
node->max_yalue *= inputj.ine [position.)*; 
npde^min^vaOiUe = i^put^line£positioh*l] ; 
• ) else i 

uo<fe->niaxjvQluc *= i-nput^,liticr £ positional. J ; 
^ node->min.value s= -inpMti:line[p6iiti6ni]*; 

node^average = (node->max_value + node->mii^value) y 2; 
) elseM 

node->cnildl ~ make_jnodes { number /2 # position) i 
. node->oh.ild2 inake^nodes / position •+ rii2mbeX"'/2 ) ; 

'.>'■- 

retixm (node) ; 

)' 

eret_stats(node_element *node) ( 
if . (node->chiXdl != Ntfli)- { 
ge-c^stats (node-xinildl) ; 
get_sfcats (node->rfiild2) ; 

node->average = (node-> c hildl->average + node->child2->averagc) / 2.0; 
if (node->chjLldl~>inajQ_value > -nod^->child2->max_value) { 

node->max_valuG = nod«->cni_ld.l->iiiax_Lvalue; 
) else. { 

node->m«uQ_vaiue = node->child2->max value: 
) .••'''."*" ' ' 

if (notifi - >chi 1 dl - >nu-ix_va 1 ue"< nodQ->cHild^^^miru_valueV f 

node->mii\ L jva-lU€r— nod«- >cnildl^>min^_va i6 lTue/;^ . 
> else { 

node->iain_^va-lue= nodc*->child2->inin va»iue»; 

retuxnO; 

> ... 

returnO r 
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initialisers true cures ( node^el emen t w top„tiode) { 
top_jnode = maJce_nodes (nuinber_of_eleineni:s;, 0) ? 

> 

void convolve (node_elemftrtt *nod<*X # node_eleroent *nOde2) { 
i»t 

iStart, . 
iEnd, 

/* dezoarkation of resultant convolution regions */ 
spanZ,en = nodel->length node2->±ength - 1, 
constLen = aJbs (nodel-?-length - node2-> length) + 1* 
slopeLen « ( spanLen - constLen) , 
xConvStart - nodel-^poaition i- node2->position, 
xConvEnd + xConvStart spanLen; . 

float 

valFroducc - nQdel^>average_value * node2->averagejv?Llue; 

overlaptien 0 ; /* zero to catch noae-cveirlap cases */ 
if (acConvStarfc -<= 0) { 

if (jcConvEnd >- noxnEl ementfl ) 

oj/erlaplien ■ numElemencs; 

else 

overlapLen = xConvEnd? 
) else { 

if (xConvEnd >- nvunElements ) 
overlapLea = numElemtots - jxConyStart; 
else' 

ovcrlapLen = spaxiLen? 
/ * atcuxriulate left slope */ 

iStart . < xConvStajrt < 0) ? 0 z xConvstart; 

iEnd. = xConvStart slopeLcn > nwaEleaentc } ? nuinElGmcnt 3 : jcConvS c&at ti -r 
elopel*en ? . ■ 

for(i»Jl5tart; i<iEnd * i++) ; 

output„line(il += (float) (i - xConvStart) *vaXPxoduct; 

/* accuinulace constant section */ . 

iStart (xjCouvStart + slopeLen < 0) ? 0 : xConvStart + slopeken; 

iEnd = xCOAVEnd - slopfislittri > TiijucnElemcm.tS ) 7 numElfcmants s jcConvEnd - EXcrp^Leri; 
for (i-iStart? i<iBnd ; i+ + ) ; 

output__line[iI += (float) tslopaI*en «- 1) *valProdiicfc; 

y* accumulate right slope */ 
iStart = ( XConvEnd - slopeDen < 0) ? 0 ! XConvEnd - SlopeLen; , 
iEnd ~ xConvEnd > numElements) ? ixumEl*»ments - xConvEnd - slopeLen; 
f or ( i=iStaxt 7 i-ciEnd ; i++) ; 

outpuci_ltne[i] C float) (xConvEnd - i) *valProdUCC; 

) • • '■• . ' - 

te:^t:_caiidiAELt;cs? (nodel # nod«2) (. 

/* testl return if the absolute level of intensity at this level is low */ 
if ( (nodel->average * node2->average ) <- averago^threshold ) . 
. return{) ; 

/* test2 return if the result: falls outside the domain of the output */ 
if ( (nodel->position - node2->position - half — length) > nuzuber^Of ^elements ) 
return ( ) 7 
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if ( (nodel*>position + nodeJ->length .+ nbde2->position + n©de2-p-len^tii - 
half-length) -c 0) 
returnO; 

/* ties £3 convolve directly if the variance is low */ 
i£ (nodel->miiuvalwe < node2 ->rain_ value ) ( 
. rai ci_v -a. 1uq-= node 1^ >xui n^yji-lu e> 

> else £ 

miiuvalue = node2->miix - value; 

) 

if (nCH3el^>max w .value > node2->nKue.value ) { 
max^vaXue - nodel*- >max^value; 

> else { 

znax_value — node2->jnaa£_yalue; 

} 

if ( (max-JvalMe - min^value) > variance_tiireshold) { 
convolve (nodel , node? ) ? 
return ( > ; 

) 

if (nodel->childl 1» NULL) C 

teg t_cr*nrH da tea ( nodel- xshildl „ node2 ) ; . 
t est_candi dates (nodel ->cnild2 , node2 ) ; 

.) 

if (node2->cliildX 1= NUUCj) { 

te$t.candj da tea ( nodel, node2->ghi.ldl) ; 
teat, Candida tes (nodel . node2 ->child2 ) ; 

> ■*■'-.■: 
r©turn() ; 

} 

main!) { 

iriiti^iseL^structur/es ( top^,nodel>) 7 . 
initialisers tractures ( top_node2,J f 
tast^candi<3dtes ( top jiodel ^ top_LHOde2*)' ; 

} 
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^t^stats (node_ei&t*e*ic 'node; ( 

g&t^stats (node->childl) 
gec_5tatrs (node->child2) 

node->avera^e ^ toode-j'-crhildl.-^ave.rasre x\ode->chi Id 2 -> average) / 2.0; 
if (Tiod<=t->ch±±dl->mAX^s&lve > node -^chl 1 d2 ~>w€UK^\ralue) ( 

n ode - >.T2a^_- W 1 u e = nodo->chl ldl -^m&s^value ; 
) else ( 

node ->ih£lx_ val u e - node ->chi 1 d2->&ax_ value / 

; 

if /'noc2e->chiJdI~>/7ijLn w v^alLie < node->c:h±ld2->niixi_va.l\2iz) ( 

aod&->min^v<aluc = nod^-^^hildl - *sal upvalue ; 
J else f 

node ->jotaji_ value = node->ch£ld2 ->ilt£tt_ value/ 

; 

Initial is &__££2ru c t ujres ( nod^cl Cment *top^nod&) { 
cop^nodG - jnaJfce^dodes Ci3Uznber_of_eleniant:jsr; 0) ; 
sree^ytata feop_«ode; /. 

i. • • . • . 

i^z tlalis&^stz^c tuxes (top^oclel) ; 
initialises truccix^es (c6p_j±o<Ig2} ; 




Pa ^7 cf 13. 



an X V- v 



c>0 . *:r//> 



12-MAR-99 FRI 14:32 KEjflkU NASH & Co. FAX NO. Oltfi 324353 P. 28 



12^^£ 



ART Ltd. Company Confidentiai. 1999 
APPENDIX 3 Example constant convolution 

void ronvoive (nod^^lemGnc 'node!. no(J^eieme.ic 'siacl&2 ) { 

IScart, 
±End* 

1^ 

/+ demarkatxoa of reeuitant convolution regions */ 

constI>€>n ~ &bs (ziodel->length -r node2 T > length) + 2, 

slopeLen =' (spajib&n - constI*enJ>i>i, 

xConvSta.rt = nedel->posi hi. on + node2->jposl tzlozi r 

vaJProducc - rrocfel ->average_vaJ ue * jiodfi^-MVfera^e^ value / 

oyer-Iagpren ~ 0/ /* zero to catch none* over lap cases */ 
(xConvStart < 0) { 

S£( xConvEn d numElemehts) 
overiapLea - numRl c*mcn ts ; 

ovGrl&pLen = xconvEnd; 
} else { 

d£ (xCotrvSnd >■= uumEjienJent^r^ 

elsm 

J 

/* accumulate left slope */ 

i^Starc « /xCanv^fc«ajrfc v a; ?• 0 : xCoxivSZazrt; 

iSnd ~ xComrSt&srt + slopeli&n > numEl&iGzits)' ? numElements : xConvStart + 
slopeLan; 

£or(x±iStart; l<l£nid J ; 

ou npu t_l ine (i] « (float) (x - xConvStart) *val Product; 

eccMmulate constate seccJion */ 
iS-t^-rt = ('jcCoJZvSCr^z-t + zIopeLen < O) 7 0 zoCcnvStart + slopeJJeiz; 
lEnd xCouvEzid - slopeLvn > nuinEl emem t3) ? numElescents. : :<ConvZnd - slope^en- 
forf±=±Stairt; i<iEnd ; ; 

output_l!zi&[±] *= (floac) (slopctLen + 1) +v^lProduct; 

/* accumulate jrl&lm slope */ 
±S£art.z= iizCoxvErid - ^lopeLen < 0} ? 0 : xconvEnd - slopaL&n; 
iZnd = 2tCcnv£nd > xiivnBl events). ? izumSlezteaCS r xCojivEnd - slcpeL,Grz; 
£cr ( S ~d Sta.rZ ; l<isnd ; / 

<?«:put„ ZirzedJ fflo&c) (zcCazivBrid - JL ) *val Product; 
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APPENDIX 4 If either candidate pair is rejected or convolution is performed 
dxrectly then each candidate is not tested further with sub-hode«? 
of other candidates 

tes ^candidates (nodel, node2) { 

/* cestl if the absolute 1<*vg1 of Intensity ac thi* Jevci is low * / 

if ( (nodGX->aver&ge * nod&2 -^avxurage ) < e. m Sera&e_tIureshol d ) 
return () / 

/* cest2 return if the result falls outside the domain of the output V 

if ( <nodel-> P o3\tiox - pode2 -^position - half-length) > numr>&r_of eiements; 

retuimo : ^ 
If ( (nodel -position + nodel~> length -t node2 -position * node 2 ->lenath - 
half-length) < 0) 

/* test3 couxrolve directly If tiie variance is low */ 
if (nodel->min^value < node2-?rm±n._yzLlue ) { 

} els* ( 

xsiltL^VCklue x o de2 - >min_ val ue; 

i£ r^od^J ->inax_value > ode 2 - ->majr_ v-al u e ; 

xnax_vaJu© =^ z*odel->n:ax_vcilue/ 
J else { 

maxLVaJiie " nod02->mo^C.v^lue / 

J 

if ( (max^ value - iiu.n_ value; > vax-ia/zc:e_t^!i-eshole?J f 
con voJ ve (nodel . node2) 
retuxao / 

if (nod&l~>ahildl NULZ,) { 

cest_c^dld<ztcs(nodel~->childi,nod&2) ; 

test_ candi tes (ncdsl ->-cfti 1 <32 „ n0de2 ) / 
i ' 
i;T . (node2~x=hildl .'- NULL) { • 

tes C_cj" nff 7 <&a t gs Vnodei „ no<242->cJ*ildJLj / 

test^candid^ C« (nod&l , nodG2->ch±ld2 ) ; 
1 ■■*■'■..- 
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Hierachical Convolution 
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This Page is Inserted by IFW Indexing and Scanning 
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BEST AVAILABLE IMAGES 

Defective images within this document are accurate representations of the original 
documents submitted by the applicant. 

Defects in the images include but are not limited to the items checked: 

□ BLACK BORDERS 

□ IMAGE CUT OFF AT TOP, BOTTOM OR SIDES 

□ FADED TEXT OR DRAWING 

□ BLURRED OR ILLEGIBLE TEXT OR DRAWING 

□ SKEWED/SLANTED IMAGES 

□ COLOR OR BLACK AND WHITE PHOTOGRAPHS 

□ GRAY SCALE DOCUMENTS 

□ LINES OR MARKS ON ORIGINAL DOCUMENT 

□ REFERENCE(S) OR EXHIBIT(S) SUBMITTED ARE POOR QUALITY 



^ OTHER: (li*t Sj>s\s 



IMAGES ARE BEST AVAILABLE COPY. 
As rescanning these documents will not correct the image 
problems checked, please do not report these problems to 
the IFW Image Problem Mailbox. 
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